Papers with shallow models
A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus (2024.acl-long)
Copied to clipboard
| Challenge: | Natural language inference (NLI) is an actively studied topic serving as a proxy for natural language understanding. |
| Approach: | They propose to use a Romanian NLI corpus to analyze sentence pairs . they use multiple machine learning methods to establish competitive baselines . |
| Outcome: | The proposed model improves on the best model by employing a new curriculum learning strategy based on data cartography. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
ReInceptionE: Relation-Aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph Embedding (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods are limited in their expressiveness and lack structural information in the embeddable space. |
| Approach: | They propose to use a relation-aware network to learn query embedding . they first explore the Inception network to further increase interactions between head and relation embedders . |
| Outcome: | The proposed network improves performance on WN18RR and FB15k-237 datasets. |
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning and CoT are still poorly understood, but the precise mechanisms and architectural factors driving ICL and Co T are still unclear. |
| Approach: | They propose a framework and methodology to generate synthetic tokenized datasets and study chain-of-thought (CoT) in-context learning in language models. |
| Outcome: | The proposed framework and methodology allows fine grained control over the complexity of in-context examples by decoupling causal structure from underlying token processing functions. |
Less Is MuRE: Revisiting Shallow Knowledge Graph Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graphs encode knowledge in the form of subject-predicate-object triples, which is notoriously incomplete. |
| Approach: | They propose a framework for analyzing existing shallow knowledge graph models and their extensions. |
| Outcome: | The proposed framework shows that MuRE and ExpressivE are highly competitive . it can capture the same class of rule bases as state-of-the-art region-based embedding models. |